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^ We carry out a delay stability analysis (i.e., determine conditions under which expected 

steady-state delays at a queue are finite) for a simple 3-queue system operated under the Max- 
Weight scheduling policy, for the case where one of the queues is fed by heavy-tailed traffic (i.e, 
^0 when the number of arrivals at each time slot has infinite second moment). This particular 

Q system exemplifies an intricate phenomenon whereby heavy-tailed traffic at one queue may or 

may not result in the delay instability of another queue, depending on the arrival rates. 

^ While the ordinary stability region (in the sense of convergence to a steady-state distribution) 

is straightforward to determine, the determination of the delay stability region is more involved: 
(i) we use "fluid-type" sample path arguments, combined with renewal theory, to prove delay 
instability outside a certain region; (ii) we use a piecewise linear Lyapunov function to prove 
delay stability in the interior of that same region; (iii) as an intermediate step in establishing 

^ delay stability, we show that the expected workload of a stable M/GI/1 queue scales with time 

as 0{t^^^^~^'^^), assuming that service times have a finite 1 -|- 7 moment, where 7 € (0, 1). 
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1 Introduction 



We consider a simple 3-queue system operated under the Max- Weight scheduhng poHcy, for the 
case where one of the queues is fed by heavy-tailed traffic (i.e, when the number of arrivals at each 
time slot has infinite second moment). The performance metric that we are interested in is delay 
stability: a queue is delay stable if the expected steady-state delay in that queue is finite, and 
delay unstable otherwise. 

Under the Max- Weight scheduling policy, the notions of stability and delay stability coincide 
when all primitive stochastic processes are light-tailed; that is, all queues are delay stable for all 
arrival rate vectors within the stability region of the system (see, e.g., [21]). On the other hand, for 
a system of parallel queues served by a single server, it is known that if one of the arrival streams 
is heavy-tailed, then all queues are delay unstable, for all positive arrival rates |14|. The system 
that we study in this paper exhibits an intermediate behavior: we show that in the presence of 
heavy-tailed traffic, there may be nontrivial parts of the stability region where a queue is delay 
stable, and nontrivial parts of the stability region where the same queue is delay unstable. This 
exemplifies an intricate "delay instability propagation" whereby heavy-tailed traffic at one queue 
may or may not result in the delay instability of another queue, depending on the numerical values 
of the arrival rates. 

The study of queueing systems with heavy-tailed traffic is motivated by significant evidence 
that traffic in real-world networks exhibits strong correlations and statistical self-similarity over 
different time scales. This observation was first made by Leland et al. [13] through an analysis 
of Ethernet traffic traces. Subsequent empirical studies have documented this phenomenon in 
other networks, while accompanying theoretical studies have associated it with heavy-tailed arrival 
processes; see [16] for an overview. At the same time, the performance analysis of Max- Weight 
scheduling is of independent interest, with a long history and rich literature, see, e.g., [T|[6| [20|[2l] . 
The feature that makes the Max- Weight policy appealing is its throughput optimality, i.e., its 
ability to stabilize a queueing system whenever this is possible. In other words, dynamic instability 



phenomena, such as the ones reported in [12,17 , are guaranteed not to occur under Max- Weight 
scheduling. 
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The impact of heavy-tailed traffic has been analyzed extensively in the context of single and 
multi-server queues; see the survey papers [3|[5] and the references therein. However, there is little 
work on more complex queueing systems and scheduling policies. A notable exception is the work 
by Borst et al. [i] , which studies the Generalized Processor Sharing policy in the presence of heavy- 



tailed traffic. Related results, which come closer to the subject of this paper, are also given in 14 
which reports a nontrivial delay stability region for a system of parallel queues served by a single 
server under the Round- Robin policy. In contrast to the aforementioned scheduling policies, Max- 
Weight is a queue length-based policy, whose dynamics are more complex and harder to analyze. 
Finally, the work by Jagannathan et al. [ll] presents a queueing system with heavy-tailed traffic, 
and under the Max- Weight policy, with a nontrivial delay stability region. However, what affects 
delay stability in that study is the intermittent access of servers to queues, arising, e.g., in wireless 
networks. Consequently, the underlying mechanism that determines the delay stability region there 
is very different than the one analyzed here. 

The main contributions of this paper are as follows: 

(i) we show that under the Max- Weight policy, heavy-tailed arrivals at one queue may or may not 
cause delay instability at other queues, as determined by an intricate instability propagation 
phenomenon; 

(ii) we use "fluid-type" sample path arguments, combined with renewal theory, to prove delay 
instability if the arrival rate to a specific queue is greater than a certain threshold (Proposition 

2); 

(iii) we use drift analysis (over a sufficiently long time interval) of a suitably constructed piecewise 
linear Lyapunov function to prove delay stability if the arrival rate to a specific queue is less 
than that same threshold (Proposition 3); 

(iv) as an intermediate step in establishing delay stability, we show that the expected workload of 
a stable M/GI/1 queue with heavy-tailed service times scales sublinearly in time, assuming 
just the existence of the 1 -|- 7 moment of service times (see the proof of Proposition 3). 

The rest of the paper is organized as follows. In Section 2 we give a detailed description of 



the queueing system under consideration and the probabiUstic assumptions that we make, together 
with some necessary definitions and notation. In Sections 3 and 4 we prove our delay instabiUty 
and stabihty results, respectively. We conclude in Section 5 with a brief discussion and future 
directions of research. 

2 Model and Definitions 

Throughout the paper we denote by Z+ and N the sets of nonnegative and positive integers, 
respectively. Similarly, the Cartesian product of M copies of Z+ is denoted by . We use 1e 
for the indicator variable of event E. Finally, stands for max{x, 0}, the nonnegative part of a 
scalar x. With few exceptions, we follow the usual convention of using lower case letters to denote 
real numbers or vectors and upper case letters to denote random variables or events. 

2.1 The basic model and dynamics 

We consider the queueing system depicted in Figure 1, consisting of three single-class, single-server 
queues, with infinite capacity. Time is slotted and traffic consists of constant size packets. We 
define Ai{t) as the (nonnegative) number of packets (a random variable) that queue i G {1,2,3} 
receives at slot t G Z+, and let A{t) = {Ai{t) , A2{t) , As{t)) . We assume that arrivals occur at the 
end of the time slots. We define Qi{t) as the number of packets in queue i at the beginning of time 
slot t, and let Q{t) = (Qi(t), (52(t), QsW)- 

We define Si{t) as the number of packets that the servers attempt to remove from queue i 
during time slot t, and let S{t) = {Si{t), S2{t), S3{t)). We assume that Si{t) can only take values 
in {0, 1}. An attempt will be successful if and only if queue i has a packet, so that the number 
of packets actually removed from queue i during time slot t is Si{t) ■ l{Qj(t)>o}- Accordingly, the 
queue length dynamics take the form 

Qi{t + l) = Qiit) + Ai{t)-Si{t)-l{Q^^t}>o}, teZ+, iG{l,2,3}, 
initialized with an arbitrary vector of initial queue lengths Q{0) in Z^. 
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Figure 1: A simple system of three single-class, single-server queues. A scheduler decides, at each 
time slot, whether queues 1 and 2 will be served simultaneously, or whether queue 3 will be served 
alone. Queue 1 receives heavy-tailed traffic, whereas queues 2 and 3 receive light-tailed traffic. 

2.2 Scheduling constraints and the Max- Weight pohcy 

We assume that the servers of queues 1 and 2 can be active simultaneously, whereas the server of 
queue 3 can only be active alone. "Scheduling constraints" of this type are common in communica- 
tion networks, such as wireless networks or data switches. Formally, the sets of queues {1, 2} and 
{3}, as well as the empty set, are called the feasible schedules. Furthermore, S{t) is constrained 
to take values in the set 5 = |(0, 0, 0), (1, 1, 0), (0, 0, for all t G The service discipline of 
packets within each queue is assumed to be "First Come, First Served." 

The choice of a service vector S{t) G 5 is made at each time slot according to a scheduling 
policy. In this paper, we focus exclusively on the Max- Weight policy, which chooses a schedule 
with the maximum total workload, at each time slot. Formally, 

3 

S{t) G argmaxj ^Qi(t) • S'ij, t G Z+. 

^^'^ i=l 

In case of a tie, each one of the feasible schedules is chosen with equal probability. 
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2.3 Assumptions on the eirrival processes 

We assume that the discrete time stochastic arrival processes {Ai{t); t G ^+}, i G {1)2,3} are 
independent and identically distributed (IID) over time, and mutually independent. We denote 
by Aj = £^[^i(0)] the arrival rate of the i*^ process, and let A = (Ai, A2, A3) For the model to be 
interesting, all rates are assumed to be strictly positive. 

We say that a random variable X is heavy-tailed (respectively, light-tailed) if E[X^] is infinite 
(respectively, finite) . Similarly, we say that the z*'* arrival process is heavy or light-tailed depending 
on whether E[^|(t)] is infinite or finite, respectively. Throughout the paper we assume that the 
first arrival process is heavy-tailed, and the other two arrival processes are light-tailed. 

2.4 The stabihty region 

A batch of packets arriving to a queue at any given time slot can be viewed as a single entity, 
and will be referred to as a file. We define the end-to-end delay of a file to be the number 
of time slots that the file spends in the system, starting from the time slot right after it arrives, 
until the time slot that its last packet exits the system. (For example, a two-packet file that arrives 
during time slot t and is served during the next two time slots has an end-to-end delay of 2.) 
For A; G N, we denote by Di{k) the end-to-end delay of the k^^ file arriving at queue i, and let 
D{k) = {Di{k),D2ik),D3{k)). 

The following definition gives the precise notion of stability that we use in this paper. 

Definition 2: (Stability) The earlier defined queueing system is stable under a particular 
scheduling policy if the vector- valued sequences {Q{t); t G Z+} and {D{k); A; G N} converge in 
distribution, and their limiting distributions do not depend on (5(0). 

Notice that our definition of stability is slightly different than the commonly used one (posi- 
tive recurrence of the Markov chain of queue lengths), because it includes the convergence of the 
sequence of file delays {D{k); k G N}. The reason is that we are interested in properties of the lim- 
iting distribution of {D{k); A; G N} and, naturally, we need to ensure that this limiting distribution 
exists. 
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An arrival rate vector A is said to belong to the stability region of the system, if there exist 
nonnegative real numbers ^12 and /is, such that 

max{Ai, A2} < fJ.12, 
A3 < 
IJ-u + Us <l- 

Intuitively, one can think of fii2 and /X3 as the fractions of time that the schedules (1,1,0) and 
(0, 0, 1), respectively, are applied. If a positive arrival vector A is outside (and at a positive distance) 
from the stability region, it is easily shown that there is no scheduling policy that makes the system 
stable. In contrast, when A belongs to the stability region, the system can be made stable, e.g., by 
using the Max- Weight policy; see Lemma 1 below. Because of these considerations, we will assume 
throughout the rest of the paper that A belongs to the stability region. 

Lemma 1: (Stability under Max- Weight) The queueing system under consideration is 
stable under the Max- Weight scheduling policy, for all positive arrival rate vectors in the stability 
region. 

Proof. It can be verified that the sequence {Q{t); t € Z+} is a time- homogeneous, irreducible, 
and aperiodic Markov chain on the countable state-space Z'^. Proposition 2 of ^20j implies that 
this Markov chain is also positive recurrent. Hence, {Q{t); t £ Z+j converges in distribution, 
and its limiting distribution does not depend on Q{0). Based on this, it can be verified that 
the sequence {D{k); k £ N} is a (possibly delayed) aperiodic and positive recurrent regenerative 
process. Therefore, it also converges in distribution, and its limiting distribution does not depend 
on Q(0); see [l9]. □ 

We let Q = {Qi, Q2, Q3) and D = {Di, D2, D3) be random vectors distributed according to the 
limiting distributions of {Q{t); t G and {D{k); k G N}, respectively, under the Max- Weight 
scheduling policy, and refer to them as the steady-state queue lengths and delays. 
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2.5 Delay stability 

We now define formally, and then discuss, the property that we will be focusing on. 

Definition 3: (Delay Stability) Queue i G {1, 2, 3} is delay stable under a particular schedul- 
ing policy if the system is stable under that policy, and E[L>j] is finite; otherwise, queue i is delay 
unstable. 

The following lemma relates the steady-state quantities E[Qj] and IE[L'j], and will help us prove 
delay stability results. 

Lemma 2 |15|: For the system under consideration, and under Max- Weight scheduling. 



In the remainder of the paper we will focus on the delay stability of queue 2. This is because 
under the Max- Weight scheduling policy, queues 1 and 3 are always delay unstable, as established 
formally in Theorems 1 and 2 of the companion paper [15] . 

Proposition 1 |15|: For the system under consideration, and under Max- Weight scheduling, 
queues 1 and 3 are delay unstable, for all positive arrival rate vectors in the stability region. 

We provide some intuition as to why Proposition 1 is true. For queue 1, the result follows easily 
from the Pollaczek-Khinchine formula for the expected delay in a stable M/GI/1 queue, and a 
stochastic comparison argument. Regarding queue 3, we argue as follows. Queue 1 is occasionally 
very long (infinite, in steady-state expectation), due to the heavy-tailed nature of the traffic that 
it receives. Whenever queue 1 becomes very long, queue 3 is starved and builds up, at a more or 
less constant rate, to a size comparable to that of queue 1. Thus, large values of Qi{t) result in 
large values of Qsi^t) at subsequent times. As E[(5i] is infinite, we can argue that ^[Qa] is infinite 
as well, and Lemma 2 implies that queue 3 is delay unstable. 

Our main results will concern queue 2. We will show in the next two sections that it is delay 
unstable (respectively, delay stable) if A2 is greater (respectively, less) than (1 + Ai — A3)/2. 




iG {1,2,3}. 
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3 Delay Instability Region 

Lemma 1 and Proposition 1 in the previous section establish that the queueing system under 
consideration is stable, and that queues 1 and 3 are delay unstable for every positive arrival rate 
vector in the stability region. Regarding queue 2, the situation is more complicated. One could 
argue that queue 1, by virtue of its receiving heavy-tailed traffic, would tend to claim service more 
often, and that queue 2 would benefit from the resulting additional servive opportunities. Somewhat 
surprisingly though, there is a more intricate indirect mechanism at play that can render queue 2 
delay unstable. 

Proposition 2: For the system under consideration, with positive arrival rates within the 
stability region, and under Max- Weight scheduling, if the arrival rates satisfy A2 > (1 + Ai — A3)/2, 
then queue 2 is delay unstable. 

Before proceeding with the formal proof of Proposition 2, we provide an intuitive outline of the 
argument, also aimed at explaining the threshold value (1 + Ai — A3)/2. Our approach is based on 
tracking the evolution of the system on a particular set of "fluid" sample pathsj^ assume that at 
time slot 0, queue 1 receives a very large file, consisting of b packets. For a long period of time 
after that, queue 3 does not receive service under the Max- Weight policy, and builds up. If the 
arrival processes of all traffic fiows are close to their "average behavior" (in the Strong Law of Large 
Numbers sense), then at the time slot when the service switches from schedule {1,2} to schedule 
{3}, the lengths of both queues 1 and 3 are proportional to b, whereas queue 2 is still small. From 
that point on, the Max- Weight policy will drain the weights of the two schedules at roughly the 
same rate, until one of the weights becomes zero. 

Let fii be the average departure rate from queue i during the latter period. For the weights of 
the two schedules to be drained at the same rate, the departure rates have to satisfy: 

Al + A2 - /il - /U2 = A3 - H3. 

^In essence, our argument employs a "fluid approximation"; however, instead of establishing formal convergence 
to a fluid model (which would be unduly complicated for our purposes), we proceed in an elementary manner, arguing 
from first principles. 
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Moreover, the fact that Max- Weight is a work-conserving pohcy imphes that 

Ml + = 1- 

Finally, since queues 1 and 2 arc served simultaneously, and queue 2 may be empty through parts 
of the draining period, we have that 

Ati > IJ'2- 

The above equations and some simple algebra imply that 

^ 1 + Ai + A2 - As 
/^2< ^ . 

Suppose that the arrival rates satisfy 

1 + Ai - A3 
A2> ^ . 

Then, 

^ ^ 1 + Ai + A2 - A3 ^ 

A2 > ^ > IJ-2- 

This implies that queue 2 builds up at a roughly constant rate, to size 0(6), during a period of time 
whose duration is proportional to b, and the integral of Q2 over a busy period of the process becomes 
of order 0{b^). Because b is drawn from a heavy-tailed distribution (infinite second moment), it 
follows that E[Q2] is infinite. 

Proof. We start by defining a shorthand notation that we will be using in the course of the proof. We 
say that a random variable X scales at least linearly with b on the event H, and write X = fijj (6), if 
there exist positive constants k and k' (possibly depending on the event H), such that X > k-b—k', 
for all sample paths in H. 

We break the proof into four steps, which follow the various stages in our earlier proof outline. 

Step 1: buildup of queue 3 following a large arrival to queue 1. 

Because the system is stable, it empties infinitely often, and the times at which Q{t) = are renewal 
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epochs. Let us consider the system at a typical renewal epoch which, for simplicity of notation, we 
assume to happen at time 0. Consider the set of sample paths for which, at time slot 0, queue 1 
receives a file that consists of b packets, and all other queues receive no traffic; we denote this set 
of sample paths by H{b). Since Aj < 1 (stability), and since 1 - P(A(0) = 0) = P(A(0) > 0) < 
E[Ai(0)] = Xi <1, we have f{Ai{0) = 0) > 0, for all i. Let B be the support of the distribution of 
Ai{0). Using the independence of the arrival processes, we have, for every b e B, 

¥{H{b)) = P(Ai(0) = b) ■ P(yl2(0) = 0) • F{As{0) = 0) > 0. 

For sample paths in H(b), we denote by the first time slot, starting from 0, when the length 
of queue 3 becomes greater than or equal to the sum of the lengths of queues 1 and 2: 

= mm{t >0\Q3{t)> Qi{t) + Q2{t)} ■ luip)- 

The first part of the proof is to show that Qi{T^) and QziT^) scale at least linearly with 6, provided 
all arrival processes are close to their "average behavior." 
By the definition of the stopping time , we have 

Qi{Tl) < QiiTl) + Q2{Tl) < Q^{Tl). 

A direct consequence of the Strong Law of Large Numbers is that for every e > there exists (5 > 0, 
such that the set of sample paths 

t 

A = i^{\i-e)t-5<^Ai{T)<{\ + e)t + 5, Vt e N, ViG {1,2,3}}, 

T=l 

has positive probability. Consequently, the set of sample paths 

t 

^{b) = [{\i-e)t-5<Y,Mr)<{h + e)t + 5, e {I, . . . - l), VzG{1,2,3}}, 

T=l 

has positive probability, uniformly over all b. 
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Prom now on we fix a small e > 0. (How small it will have to be will become apparent in the 
course of the proof.) We then fix a corresponding S such that inf{,P(A(6)) > P(A) > 0. Let H{b) 

be the set of sample paths H{b) n A{b), and observe that H{b) n A(6) D H{b) n A. Then, the IID 
nature of the arriving traffic implies that H(b) and A are independent, so that 

f{H{b)) = F{H{b) n A(6)) > ¥{H{b) n A) = ¥{H{b)) ■ P(A) > 0. 

Recall that at most one packet can be removed from each queue at any given time slot. So, for 
sample paths in H{b), we have that 

Qi{Tl) > b - {Tl - 1) + (Ai - e) • {Tl -I) -8. 

Moreover, queue 3 receives no service before time slot under the Max- Weight scheduling policy, 
which implies that 

Tl-l 

Q:,{Tl) = Mt) < (As + e) • " 1) + 
t=i 

Since Qi{T^) < Q3{T^), the last two inequalities and some algebra yield 
(1) Tl-1> 



1 + As - Ai + 2e 



(This argument requires the last denominator to be positive. This will be the case as long as e has 
been chosen small enough.) Therefore, 

(2) QsiTl) = Mt) > (As - e) • {T,' - I) - S > {X3 - e) ■ , , , - <5, 



1 + As - Ai + 2e 



which implies that QsiT^) = ^^(ft)(^')) since we can chose e to be less than As- 

Coming to queue 2, it can be verified that for sample paths in A (and hence in H{b)), and for 
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any subinterval {tq, . . . , ti} of {1, ... , T^}, 

Tl-l 

J2 Mt) < (A2 + e) • (ri - To) + 2eTo + 2S. 

t=To 

We assume that e has been chosen so that A2 + e < 1. Recall also that queue 2 gets served 
whenever it is nonempty throughout the period {1, . . . ,r^^ — 1}. We use Lindley's formula, the 
above upper bound on the arrivals to queue 2, and the fact that since queue 2 gets served whenever 
it is nonempty throughout the period {1,. . . — 1}, to conclude that 

(3) Q2{T^) < A2{Tl - 1) + 2e{Tl -l) + 26<\2 + Ae{Tl - 1) + AS. 

This shows that, essentially, Q2{Tl) does not scale with h. 

We finally turn our attention to Q\{Tl). By definition of the stopping time T^, 

(4) Q^{Tl - 1) < Qi{Tl - 1) + Q2{Tl - 1). 

By arguing similar to the derivation of Eqs. (2) and (3), it can be verified that 

(5) Q2{Tl - 1) < A2 + Ae{Tl - 2) + 4<5, 

and that Qz{Tl - 1) = ^H{b)i^)- Then, Eqs. (4) and (5) readily imply that QiiT^) = ^H{b)i^)^ 
when e is chosen sufficiently small. 

To summarize, at time slot T^, and for sample paths in H{b), the lengths of queues 1 and 3 are 
proportional to b, while queue 2 is still small. 

Step 2: draining down until queue 1 or 3 empties. 

Let be the first time slot after that either queue 1 or queue 3 becomes empty: 

= imn{t > Tl \ Q^{t) ■ QsW = 0} • l^^^^. 
We will show that if the arrival processes stay close to their "average behavior" (i.e., the event 
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H{b) occurs), then, at time slot T^, the length of queue 3 cannot be much larger than the sum of 
the lengths of queues 1 and 2. The reason is that Qi{t) + Q2{t) and Qsit) are kept roughly equal 

by the Max- Weight policy throughout the interval [T^,T^]. 

For the same constants e and 6 as Step 1, the set of sample paths 

t 

A'{b) = [{Xi-e)t-S< J2 MT)<{Xi + e)t + S, e {T^, . . . - 1}, ViG {1,2,3}} 

contains A. Let H(b) = H[h) n A'(6). The events A(6) and A'(6) arc determined by the arrivals 
over disjoint time intervals. Hence, due to the IID nature of the arrival processes, A(6) and A'(6) 
are independent. Since they both contain the positive probability event A, we have 

P(^(6)) > P(i?(6)) • (P(A))^ > 0. 

We will show that for sample paths in H{h), 

(6) Q^{Ti) < Qi{Ti) + Q2iT^) + 2e(r2 - T^) + 26 + 3. 

We first notice that queues 1 and 3 cannot empty at the same time slot, since they cannot 
be served simultaneously. Therefore, we have two possible cases: either Q^(T^) = 0, in which 
case Eq. (6) is trivially satisfied, or Qi{T^) = 0, which we henceforth assume. In the latter case, 
SiiT^ - 1) = S2iT^ - 1) = 1, and SsiT^ - 1) = 0. For sample paths in H(b), we have that 

(7) QsiT^) = QaiT^ - 1) + A^iT^ - 1) < Q^{T^ _ i) + A3 + 2e • (T^ - T^) + 26. 

Moreover, under the Max- Weight scheduling policy, and in order for the for the set of queues {1,2} 
to be served at time slot — 1, 

(8) QsiTi - 1) < QiiTi - 1) + Q2{Ti - 1). 
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Finally, 



(9) Qi{Ti - 1) + Q2{Ti - 1) - 2 < Qi(T2) + Q2{Ti). 

Eq. (6) follows immediately, by combining Eqs. (7)- (9) and using also the fact A3 < 1. 

Step 3: growth of queue 2. 

At time T^, Q^{T^) cannot much larger than Qi{T^) + Q2{T^). Therefore, queue 3 must have 
been receiving a certain fraction of the total service between times and T^. We will show that 
this results in queue 2 not receiving enough service, and that Q2 starts growing. In particular, if 
A2 > (1 + Ai — A3)/2, then, for the sample paths of interest, Q2{T^) = ^^((,)(^)' 
By definition, 

Q3{Tl)>Q,{Tl) + Q2{n'). 
By subtracting the two sides of this inequality from Eq. (6), we get 

(10) QsiT^) - QsiTl) < Q,{T^) - Q,{tI) + Q2{T^) - Q2{tI) + 2e{T^ - T^) + 25 + 3. 
For sample paths in H{b), define the random variables 

1 



Mi 



which are the average service rates to each queue during the interval {T^, . . . — 1}. Notice that 
Ml = M2 and Ati + /X3 = 1. 

Since both queues 1 and 3 are nonempty during the inerval {T^, . . . , — 1}, we have 

(11) Qi{Ti) - Qi(r,i) < (Ai + e - w) • {tS - Tl) + S, 

(12) Q3{T^) - Q3{T^) > (A3 - e - M3) • {T^ - Tl) - 5. 
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Eqs. (10), (11), and (12) imply that 



{Xs-e-^l^)■{Ti-^')-S<{X, + e-^il)■{Ti-T,') + ^ 

+ Q2{T^) - Q2{Tl) + 2e(r2 - Tl) + 2(5 + 3. 

We replace /X3 by 1 — /ii and collect terms, to obtain 

„ (T-^ Tl^> / 1 + A1-A3+46 N 2 , Q2(T,1)-Q2(T,^) 4<5 + 3 

-/ii-(r, -r,)>-(^ ^ j.(r, -r,) + ^ — 

/l + Ai^A3 + 4ex 2 Q2(?f) 45 + 3 

-"I 2 J'^^^ 2 2~- 

For sample paths in H{h), we use the definition of A' (6) to upper bound the number of arrivals to 
queue 2. We also use the fact that queue 2 has fi2{T^ — T^) service opportunities, with H2 = f^i, 
and obtain 

Q2(TS)>(\2-f^-K}-(T;-Tl)-S 

>(,,_l±4^_3e).(T^-T;)-^-^. 



Therefore, 



Q2{T^) > I • (A2 - l±^i-^ - 3e) . (T^ - Tl) -26-1. 
If A2 > (1 + Ai — A3)/2, the constant e can be chosen sufficiently small so that 

1 + Ai - A3 o ^ n 
A2 3e > 0. 

A final observation is that the duration of the interval {T^, . . . — 1} is bounded from below by 
m.m{Qi{Tl) , Q3{TI)} , because both queues are served at unit rate. Therefore, — = ^^^(5)(&) 
and 

(13) Q2(r2) = f^^(,)(6). 
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Step 4: the growth scenario for queue 2 imphes large average queue size. 

In Step 3 we showed that for sample paths in H{b), queue 2 builds up to the order of b. We use 
this fact, and renewal theory, to show that the steady-state expected length of queue 2 is infinite. 
The sequence of times at which Q{t) = are renewal epochs. Denote by Xi the length of the 

inter-renewal period. The random variables {Xi; i G N} can be viewed as IID copies of some 
nonnegative random variable X, with finite first moment; this is because the empty state is positive 
recurrent under the Max- Weight policy (see Proposition 2 of [20j). 

We define an instantaneous reward on this renewal process: 



R^^it) = mm{Q2it),M}, t G Z+, 



where M is a positive integer. 

Eq. (13) implies that there exist positive constants c and 6o, such that 

Q2iT^) >cb, V 5 > 6o, 



for all sample paths in H{b). 

Since at most one packet from queue 2 can be served at each time slot, the length of queue 2 
is at least cb/2 packets over a time period of length at least cb/2 time slots. Hence, the aggregate 
reward R^gg, i.e., the reward accumulated over a renewal period, satisfies the lower bound 



RnL ■ l/h>h„l • > miu 



(by 



Then, the expected aggregate reward is bounded below by 



E[<J > (P(A))' • P(^2(0) = 0) • P(^3(0) = 0) • X^min 



6=1 



So, there exists a positive constant c' such that 



(14) 



c'-E 



mm 



1 



{b>bo} 



.p(Ai(0) = b). 



{0)>bo} 



< E\R. 



M 1 
aggi 
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We have argued that inter-renewal periods have finite expectation, and, clearly, the expected 
aggregate reward is finite. Then, the Renewal Reward theorem (e.g., see Section 3.4 of ^) implies 
that 



(15) = hm 1 y R''{t), w.p.l. 

Also, the fact that the reward is a bounded function of an ergodic Markov chain implies that 

1 ^"^ 

(16) hm - V R^'it) = E[min{Q2, M}], w.p.l. 

T^oo I ^ — ' 
i=0 

Eqs. (14)-(16) imply that 

I (^^^y . l^^^(o)>,„^, M^}] < E[min{Q2, M}]. 



' E 



mm • 



Taking the limit as M goes to infinity on both sides, and using the Monotone Convergence theorem 
(e.g., see Section 5.3 of (24)) gives 



^•E[Af(0)-l|^,(0)>,„|]<E[Q2]. 



Finally, the fact that the random variable vli(O) is heavy-tailed, implies that E[(52] is infinite. 
Combined with Lemma 2, this gives the desired result. □ 



4 Delay Stability Region 

In this section we establish that when A2 < (1 + Ai — A3)/2, then queue 2 is delay stable. Notice 
that this result involves the same threshold as in Proposition 2, and we therefore have an exact 
characterization of the delay stability region. 

The proof of this result relies on drift analysis of a suitable Lyapunov function. The usual 
analysis of the Max- Weight policy involves a quadratic Lyapunov function [21] . Different types of 
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Lyapunov functions have also been used for stability and performance analysis of other queueing 



networks and policies, e.g., piecewise linear functions [2j[7], and norms [18,23 . However, in the 
presence of heavy-tailed traffic, the expected value of all these Lyapunov functions in steady-state 
is infinite, which renders their drift analysis uninformative. 

Here we consider a carefully constructed piecewise linear Lyapunov function that is not radially 
unbounded (i.e., the value of the Lyapunov function need not be large when the queue lengths are 
large), and which has a negative drift only when A2 < (1 -|- Ai — A3)/2. A common technical 
difficulty with piecewise linear Lyapunov functions is that the stochastic descent property is often 
lost at locations where the Lyapunov function is nondifferentiable. This difficulty can be handled by 
either smoothing the Lyapunov function (e.g., as in [Tj), or by showing that the stochastic descent 
property still holds if we look ahead a sufficiently large number of time slots (e.g., as in [22|). We 
follow the second approach. 

Furthermore, we also assume that light-tailed traffic has exponentially decaying tails, which 
allows us to exploit the tools in [Toj. Formally, a nonnegative random variable X is exponential- 
type, if there exists some > such that E[exp(0X)] is finite]^ 

Proposition 3: For the system under consideration, suppose that the arrival rate vector 
is positive and within the stability region, and that the Max- Weight policy is used. Suppose 
furthermore that (i) ^2(0) and ^3(0) are exponential-type, and (ii) there exists some 7 > such 
that E[^}+^(0)] < 00. If the arrival rates satisfy A2 < (1 + Ai — A3)/2, then queue 2 is delay stable, 
and the steady-state length of queue 2 is exponential-type. 

Before we proceed with the formal proof of Proposition 3, we provide the underlying intuition 
by arguing (loosely) in terms of a fluid approximation. Our analysis rests on drift analysis of the 
following piecewise linear Lyapunov function 

(17) V{t) = 'iQ2{t)+[Q^{t)-Qi{t)-Q2{t)Y , teZ+. 

Our goal is to establish a negative drift for V{t), which, combined with the special structure of the 



^ We conjecture that the statement of Proposition 3 is true for any type of hght-tailed traffic. However, the results 



in 10 cannot be invoked without the additional assumption of exponential-type traffic. 
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problem (the arrivals to queues 2 and 3 being exponential- type), will imply that 



E 



3Q2 + [Qs -Qi- Q2\ 



< 00, 



where the expectation is taken with respect to steady-state distributions. Together with Lemma 2, 
this will imply the delay stability of queue 2. 

The suitability of this Lyapunov function can be seen as follows. If Qi{t) + Q2{t) > Qsit), 
then the Lyapunov function reduces to 3Q2{t), which can be easily shown to have negative drift 
(as long as Q2 > 0), because queue 2 is served under Max- Weight in that region. If, on the other 
hand, Qi{t) + Q2{t) < Qsit), then the Lyapunov function reduces to 2Q2{t) + Q-i{t) — Qi{t). In 
that region, queue 3 is served under Max- Weight, so the drift of V{t) is equal to 2X2 + A3 — Ai — 1, 
which is strictly negative by our assumption on A2. See Figure 2 for a geometric interpretation. 

Q2 




Q1+Q2-Q3 

Figure 2: A two-dimensional representation of the dynamics of the system when A2 < (1+Ai — A3) /2. 
The dashed black arrows represent the gradient of the queue length vector under the Max- Weight 
policy, and the dashed orange arrows represent the gradient of the Lyapunov function, in different 
regions of the state space. The solid red arrows represent typical trajectories of the associated fluid 
approximation. Our Lyapunov function has been chosen so that an obtuse angle is formed between 
the two in all regions of the state space, establishing a uniform negative drift over the entire state 
space. 



The analysis becomes subtler at the boundary between the two regions of the state space, 
namely when the weights, Qi{t) + (52 (i) and Qsit), of the two candidate schedules are equal. Once 
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at the boundary, the Max- Weight pohcy keeps the state (of the fluid model) there. Similar to our 
discussion of Proposition 2, the resulting departure rate from queue 2 satisfies 



Al + A2 - /il - /U2 = A3 - /is. 



Using also the properties = /"i = 1 — /U3, and our assumption on A2, it follows that the drift at 
the boundary is negative, driving the state to zero. 

Proof. Consider the piecewise linear Lyapunov function defined by Eq. (17) and let J-f be the c-field 
that corresponds to the history of the process until just before the arrivals at slot t; formally, J^t is 
the (T-algebra generated by Q{0), A{0), . . . , Q{t — 1), A{t — 1), Q{t). Throughout the proof we use 
F{X; A I n) and E[X; A\n]to denote F{X • U | ^) and E[X • U I respectively, where X is a 
random variable, A is an event, and is a cr-algebra on the sample space. 

Our goal is to show that for sufficiently large (but fixed) T G N, there exist positive constants 
a and e, such that 



and the desired result will follow from |10| . 

Suppose that V{t) > a. Then, we must have either Q2{t) > a/Q or Qsit) > Qi{t) + Q2{t) + a/2. 
We will derive the desired drift inequality by considering separately these two cases. We assume 
that T has been fixed to a suitably large value, and we define a by a = 6T. 



Case 1: Q2{t) > a/6 = T. 
Since at most one packet is removed at each time slot from queue 2, it is immediate that 



EV{t + T)-V{t) + e;V{t)>a Tt < 0, 



(18) 



(32(r) > 0, 



VrG{f, ...,t + r-i}. 



Moreover, under the Max- Weight scheduling policy, 



(19) 



^3(r) • l{Q3(r)>0} = ^3(r), Vr G {t, . . . , i + T - 1}. 
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Eq. (19) implies that a service opportunity is never wasted in queue 3 throughout {t,..., t+T— 1}, 
assuming Q2{t) > T. This is intuitively clear, because, under the Max- Weight policy, queue 3 will 
not be served unless it becomes at least as large as queue 2, hence positive, and a service opportunity 
will not be wasted). 
Let 

J{t) = Qi{t) + Q2{t)-Q3{t). 
Using this notation, and the queue length dynamics from Section 2, it can be verified that 

t+T-l 

{Mr) + A2{t) - Si{t) ■ l{QiM>0} - S2{t) ■ 1{Q2M>0}) 

T=t 

t+T-l 

(20) = (^3(r) - 53(r) • 1{Q3W>o}) + J{t + T)- J{t). 

T=t 

Moreover, the scheduling constraints imply that 

(21) Si(T) = 52(r), VrG{t,...,t + r-l}. 
Furthermore, under the Max- Weight policy (or, in general, under any non- idling policy), 

t+T-l 

(22) {S2{r) + S^{T)) = T. 

T=t 

Eqs. (19) and (20) imply that 

t+T-l t+T-l 

(23) Y ^Mr) + A2{t) - 5i(r) - S2{t)) < Y (Mr) - Ssir)) + J{t + T) - J(t). 

T=t T=t 

Combining Eqs. (21) and (23), we have 

t+T-l t+T-l 

(24) Y (^i(^) + ^2(t) - 2 • S2{t)) < Y (Mr) - Ss{r)) + J{t + T) - J(t). 

T=t T=t 
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Then, by taking into account Eq. (22), we get 



t+T-l 



t+T-1 



(25) 



-3- ^ S2{t)<- {l+Al{T) + A2{T)-A^{T)) + J{t + T)-J{t). 



T=t 



T=t 



Let us now examine the implications of this inequahty on the evolution of queue 2 throughout 
{t,...,t + T -1}. We have 



t+T-l 



3{Q2it + T)-Q2{t)) = S- J2 (^2(r)-52(r)-l{Q,(,)>o}) 

T=t 

= 3- {A2{t) - S2{t)) 

T = t 

t+T-l 

< - ^ (1 + Ai(r) - A^{t) - 2A2{t)) + J{t + T) - J(t), 



T=t 



where the second equality follows from Eq. (18), and the inequality follows from Eq. (25). This 
implies that 

V{t + T) - V{t) = S{Q2{t + T) - Q2it)) + [-J{t + r)]+ - [- J(i)]+ 

t+T-l 

<- Yl {l + Mr)-As{T)-2A2{r)) 

T=t 

+ Jit + T) - J{t) + [-J{t + T)]+ - [- J(i)]+ 
= -Y (l + Ai(r)-A3(r)-2^2(r)) + [J(t + r)]+-[J(t)]+. 

Therefore, 



E 



V{t + T)-V{t) ■,Q2{t)>T 



<-5-T-: 



'(Q2{t) > r I j-j) 



(26) 



+ E 



[J{t + T)]+-[J{t)]+;Q2{t)>T 



where (5 = 1 + Ai — A3 — 2A2 is positive by assumption. 

We have also assumed that there exists 7 > 0, such that E[ylj^'^'^(0)] is finite. Based on this, we 
will prove that E[ [J(t + r)]+ - [J(t)]+ I J^t]] scales sublinearly in T. This, in turn, will imply that 
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the right-hand side of Eq. (26) is negative, provided T is sufficiently large. 

By disregarding the contribution of queue 3, we can bound from above [J(r)]"'", r e {t, . . . ,t + 

T}, by the sum of the lengths of queues 1 and 2 during that interval. Moreover, whenever [J(t)]''' 
is nonzero, both queues 1 and 2 are served at unit rate. Through Lindley's recursion and simple 
calculations, it can be verified that 

t+T-l t+T-l 

(27) [J(t + T)]+-[J(t)]+< max^{ ^ (^i(r) - 1)} + max^ { ^ (^2(r) - 1)}. 

T=t+T-S T=t+T-S 

Also, the following inequality holds for all i G {1, 2}, and for all s G {1, . . . , T}: 



t+T-l 



t+T-l 



t+T-l 



(28) Y {Mr)-l) = iXi-l)s+ Yl iMr)-Xi)<\ Y (Mr) - Xi) 

T=t+T-S T=t+T-S T=t+T-S 

Eqs. (27) and (28) imply that for any fixed 7' e (0,7), 



»([J(i + r)]+ - [J{t)]+ > c ; Q2{t) >T\j't) 

t+T-l t+T-l 

<p(max^{ Y (^i(^)-l)} + i^f<^T{ ^ {A2{T)-l)}>c;Q2{t)>T\Tt) 

~ T=t+T-S 

}>|;Q2(t)>T| j-t) 



T=t+T-S 

t+T-l 



-^(i<.^t{| ^ iMr)-Xi] 

T=t+T-S 

t+T-l 

+ P(max^{| Y (^2(r)-A2)|}>|;Q2(i)>r| Ji) 



T=t+T-S 

t+T-l 



(29) 



»(mj^^{| Y iMr)-X,)\'^''}>{iy^'';Q2it)>T\^t) 

T=t+T-S 

t+T-l , , 

+ P(max^{| Y (^2(r)-A2)| ^ } > (|) ' ; Q2(i) > T | 



T=t+T- 



Notice that the sequence 



t+T-l 

{ Y iMr)-Xiy, sen], ^G {1,2,3}, 

T=t+T-S 
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is a martingale. Consequently, the sequence 



t+T-l 



E iMr)-X^ 



1+7' 



T=t+T-S 



is a nonnegative submartingale. Doob's submartingale inequality (see, e.g.. Section 14.6 of |24| 
and Eq. (29) imply that 



»([J(t + T)]+ - [J(t)]+ > c ; Q2(i) > T I J-^ 



'2 \ 1+7' 



t+T-l 

[| (Ai(r)-Ai) 

t+T-l 



(30) 



T = t 



1+7' 



1+7' 



■ lQ2(i)>T 



• 1. 



2W>T- 



Moreover, the Marcinkiewicz-Zygmund Strong Law of Large Numbers states that 



(31) 



t+T-l 

I E (^^(^) - 

T=t 

2^1/(1+7) 



0, iG {1,2,3} 



(see, e.g.. Chapter 6.10 of j9|). Eqs. (30) and (31) imply that if T is sufficiently large, then there 
exists /c > (independent of T and c) , such that 



'([J(t + T)]+-[J(t)]+>c;Q2(t)>T 



^ . k ,,1+7' 



for all c > 0. This gives 



E 



[J(t + T)]+-[J(t)]+;Q2(t)>T 



-Ft 



1+ ' °° 1 

c=l ^ 



Since 7' > 0, the latter sum converges, so there exists /c' > (independent of T), such that 



(32) 



E 



[J(t + T)]+-[J(t)]+;Q2(t) >T Ft <k' -T^ ■lQ^(^t)>T- 



1+7 
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Finally, Eqs. (26) and (32), and the fact that 7' < 7, imply that for any fixed e > 0, there exists a 
sufficiently large T, such that 



(33) 



E V{t + T)-V{t) + e;Q2{t)>T Tt < 0. 



Remark: the upper bound on [J(t)]"'" that we analyzed above can be viewed as an upper bound 
for a discrete-time stable M/GI/1 queue inwhere customers arrive in a Bernoulli fashion, and their 
service times are mutually independent and distributed identically to A\ (0) + Ai (0) . Since service 
times are heavy-tailed distributed, the expected steady-state workload in this queue is infinite 
(an immediate corollary of the Pollaczck-Khinchine formula). Combined with Fatou's lemma, this 
implies that the expected workload at time slot t goes to infinity as t increases. Eq. (32) shows that 
the expected workload goes to infinity at a sublinear rate (essentially, as 0(r^/(^+'''))), assuming 
just the existence of the (I-I-7) moment of service times. We note that Central-Limit-Theorem-type 
arguments cannot be used here, since the second moment of arrivals does not exist. 



Case 2: Q^it) > Qi{t) + Q2{t) + 3T. 



Since at most one packet is removed at each time slot from queue 3, it is immediate that 



Q3(t) > 0, 



We{t,..., t + T-1}. 



Based on this, it can be easily verified that Eq. (25) still holds. This implies that 



t+T-l 



S{Q2{t + T)-Q2{t)) = S- (^2(r)--S2(r)-l{Q,(,)>o}) 



T=t 

t+T-1 t+T-1 




<- ^ (l + yli(r)-yl3(r)-2A2(r)), 




+ j(i + T)- J(i) + 3- Y S2{r)-hQ,(r)=o}- 



T=t 
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Letting D be the event Q3(t) > Qi{t) + Q2{t) + 3T, this gives 



E 



V{t + T)-V{t) -D I Ji] < -(5-r- lB+E[[J(t + r)]+- [J(t)]+ -D 



(34) 



+ 3-e[ ^ 52(r) • l{Q,(,)=o} 



Working similar to the previous case, it can be verified that for any fixed 7' G (0,7), there exists 
/c' > (independent of T) such that 



(35) 



E 



7t 



<k' -T^+i ■ Id. 



In view of Eqs. (34) and (35) and in order to estabhsh the negative drift property of the Lyapunov 



function, it is sufficient to consider a large time horizon T, and to show that E[^*tJ~^ S'2(r) • 
l|Q2(r)=o} I ^t] scales sublinearly in T. 

In order to have a wasted service opportunity at queue 2, the schedule {1,2} must first claim 
the server. This can only happen if X^^=T~^(^i(''") + ^2(t)), the aggregate arrivals to queues 1 and 
2 during the interval {t, . . . ,t + T — 1} exceed the initial difference between the weights of the two 
schedules (which is at least 3r), minus the departures from queue 3 during the same period (which 
are at most T). It follows that 



t+T-l 



t+T-l 



T=t 



{ ^ S2{t) ■ l{Q,(.)=o} > 0} c{ ^Mr) + A2{t)) > 3r - t} 

T=t 

t+T-l 

={ Y (^i(^) + Mr) - Ai - A2) > (2 - Ai - A2)r} 

T=t 

t+T-l 

c{| J2 (^i(T)+^2(r)-Ai-A2)| > (2-Ai-A2)r}. 



r=t 



Note that Ai + A2 < 2, since the arrival rate vector was assumed to be in the stability region 
of the system. Then, using Markov's inequality and Eq. (34), we obtain that there exists ^ > 
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(independent of T) such that 



t+T-l 



E ^2(r)-l|Q,(,)=o}>0;Z) 

r=t 
t+T-l 

<P(| (^i(t)+.42(t)-Ai-A2)| >(2-\^-\2)T -D 

r=t 

E[|EttT"'(^iW + ^2(r)-Ai-A2) 



Ft 



< 



(2 - Ai - A2)T 



provided T is sufficiently large. Since the number of wasted service opportunities at queue 2 during 
an interval of length T is bounded by T, we conclude that 



(36) 



t+T-l 



T=t 



Eqs. (34)-(36) imply that for any given e > 0, there exists a sufficiently large T, such that 



(37) 



E 



V{t + T)-V{t) + e;D 



< 0. 



This completes the derivation of the drift inequality for the second case. 

To summarize, we have shown, in Eqs. (33) and (37), that for any given e > 0, we can choose 
T sufficiently large and let a = 6T to guarantee that 



E 



V{t + T) - V{t) + e ;F(t) > a 



< 0, i G 



Theorem 2.3 of |10j, and the fact that the sequence {y{t)] t G Z+j converges in distribution, 
imply that the steady-state length of queue 2, Q2, is exponential- type, and, in particular, has finite 
expectation. □ 



An immediate corollary of Proposition 3 and Lemma 2 is that queue 2 is delay stable, provided 
its arrival rate is sufficiently small. To establish this, we assumed that the stochastic processes 
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{A2{t); t G and {A^{t); t G Z+j are exponential-type. At the same time, this additional 
assmnption enabled us to prove a result that is much stronger than delay stability: combining 
Proposition 3 with the distributional Little's Law (e.g., see [25]), we have that the steady-state 
delay in queue 2 is exponential-type. 

5 Discussion 

We considered a simple queueing system with heavy-tailed traffic, and showed a rate-dependent 
delay stability phenomenon under the Max- Weight scheduling policy: there is a part of the stability 
region where a certain queue is delay stable, and a part of the stability region where the same queue 
is delay unstable. We note that this phenomenon would not arise if all stochastic primitives were 
light-tailed. 

Despite the simplicity of the underlying queueing system, the proofs of our main results (Propo- 
sitions 2 and 3) are long, somewhat technical, and, in some sense, "tailored" to the specifics of the 
system. Hence, generalizing these results using similar methods is not straightforward. What is 
missing is a systematic and practical methodology for determining which queues are delay sta- 
ble, which are delay unstable, and over which parts of the stability region, for any given switched 
queueing system. 

On a more technical side, the proof of the delay stability result (Proposition 3) requires the 
additional assumption that all light-tailed arrival processes are exponential-type. Only then is 
drift analysis of a piecewise linear Lyapunov functions meaningful (i.e., we can invoke the results 
in [To]). One possible way to overcome this restriction would be through a suitably defined piecewise 
quadratic Lyapunov function. However, drift analysis seems cumbersome in that case. 
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